Sound normalization and frequency remapping using haptic feedback

ABSTRACT

Method and devices for processing audio signals based on sound profiles are provided. A sound profile can include data related to haptic movement of the audio data which is specific to a left ear or a right ear, demographic information, ethnicity information, age information, location information, social media information, intensity score of the audio data, previous usage information, or device information. A sound profile can be customized for individual user to include the inaudible frequency range at high frequency end and low frequency end. Audio data within the inaudible frequency range can be compensated by haptic movement corresponding to the inaudible frequency range. A sound profile can further include an audio frequency range and its lowest audible volume for a user. A sound profile can be provided to a user without any action from the user&#39;s part.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/970,787, filed May 3, 2018, which is a continuation-in-part of U.S.application Ser. No. 15/669,823, filed on Aug. 4, 2017, now U.S. Pat.No. 10,560,792, which is a continuation of U.S. application Ser. No.14/609,357, filed on Jan. 29, 2015, now U.S. Pat. No. 9,729,985, whichis a continuation of U.S. application Ser. No. 14/512,679, filed on Oct.13, 2014, now U.S. Pat. No. 8,977,376, which is a continuation-in-partof U.S. application Ser. No. 14/269,015, filed May 2, 2014, now U.S.Pat. No. 8,892,233, which is a continuation of U.S. application Ser. No.14/181,512, filed on Feb. 14, 2014, now U.S. Pat. No. 8,767,996, whichclaims priority to U.S. Provisional Application 61/924,148, filed onJan. 6, 2014, which are incorporated by reference herein in theirentirety.

TECHNICAL FIELD

The present invention is directed to improving the auditory experienceby modifying sound profiles based on individualized user settings, ormatched to a specific song, artist, genre, geography, demography, orconsumption modality. The sound profile can be further related toinaudible frequency range, haptic movement, ethnicity information, ageinformation, and more.

BACKGROUND

Consumers of media containing audio—whether it be music, movies,videogames, or other media—seek an immersive audio experience. Toachieve and optimize that experience, the sound profiles associated withthe audio signals may need to be modified to account for a range ofpreferences and situations. For example, different genres of music,movies, and games typically have their own idiosyncratic sound that maybe enhanced through techniques emphasizing or deemphasizing portions ofthe audio data. Listeners living in different geographies or belongingto different demographic classes may have preferences regarding the wayaudio is reproduced. The surroundings in which audio reproduction isaccomplished—ranging from headphones worn on the ears, to inside cars orother vehicles, to interior and exterior spaces—may necessitatemodifications in sound profiles. And, individual consumers may havetheir own, personal preferences.

SUMMARY

The present inventors recognized the need to modify, store, and sharethe sound profile of audio data to match a reproduction device, user,song, artist, genre, geography, demography or consumption location.

Various implementations of the subject matter described herein mayprovide one or more of the following advantages. In one or moreimplementations, the techniques and apparatus described herein canenhance the auditory experience. By allowing such modifications to bestored and shared across devices, various implementations of the subjectmatter herein allow those enhancements to be applied in a variety ofreproduction scenarios and consumption locations, and/or shared betweenmultiple consumers, to detect inaudible frequency ranges for individualusers, to compensate for the inaudible frequency ranges, and to controlhaptic devices. Collection and storage of such preferences and usagescenarios can allow for further analysis in order to provide furtherauditory experience enhancements.

In general, in one aspect, the techniques can be implemented to includea memory component capable of storing media content and a networkcomponent capable of receiving a first sound profile, where the mediacontent can include audio data and audio metadata related to the audiodata in the media content, and the first sound profile is capable ofcontaining initial preselected parameters for modifying the audio data,one or more preselected parameters in the first sound profile can bematched to one or more pieces of information in the audio metadatarelated to the audio data in the media or the information about thedevice for reproducing enhanced media content, and the one or morepreselected parameters for modifying the audio data are received withoutpreviously transmitting a user's parameters for modifying the audio datafrom the device. In addition, the techniques can be implemented toinclude a processor component configured to access the stored mediacontent including the audio data from the memory component and theinformation about the device for reproducing enhanced media content,access the first sound profile received by the network component,determine a first inaudible frequency range that is inaudible for auser, compensate the first inaudible frequency range under a first setof compensation parameters, update the first sound profile with thefirst set of compensation parameters for the first inaudible frequencyrange to create a second sound profile, and modify the audio file in themedia content according to the second sound profile.

In general, in another aspect, the techniques can be implemented toinclude a haptic device, a memory component capable of storing mediacontent, and a network component capable of receiving a first soundprofile, where the media content includes audio data and audio metadatarelated to the audio data in the media content, the first sound profileis capable of containing initial preselected parameters for modifyingthe audio data, one or more preselected parameters in the first soundprofile are matched to one or more pieces of information in the audiometadata related to the audio data in the media or the information aboutthe device for reproducing enhanced media content, and the one or morepreselected parameters for modifying the audio data are received withoutpreviously transmitting a user's parameters for modifying the audio datafrom the device. In addition, the techniques can be implemented toinclude a processor component configure to access the stored mediacontent including the audio data from the memory component and theinformation about the device for reproducing enhanced media content,access the first sound profile received by the receiver component, playthe audio file in the media content modified according to the firstsound profile, and control the haptic device movement while playing theaudio file based on the first sound profile.

In general, in another aspect, the techniques can be implemented toinclude a haptic device with a sensor capable of being placed on acontact point on a human body, a memory component capable of storingmedia content, and a network component capable of transmittinginformation about the device for reproducing enhanced media content,audio metadata related to the audio data in the media content, and asound profile over a network, and capable of receiving a first soundprofile over the network, where the media content includes audio dataand audio metadata related to the audio data in the media content, andthe first sound profile is capable of containing initial preselectedparameters for modifying the reproduction of the audio data. Inaddition, the techniques can be implemented to include a processorcomponent configure to access the stored media content including theaudio data from the memory component, access the first sound profilereceived by the network component, select a frequency range, determine alowest haptic intensity for haptic movement that can be detected at thecontact point for the frequency range, update the first sound profilewith the lowest haptic intensity for the frequency range to become asecond sound profile, and modify the reproduction of the audio data inthe media content according to the second sound profile.

These general and specific techniques can be implemented using anapparatus, a method, a system, or any combination of apparatuses,methods, and systems. The details of one or more implementations are setforth in the accompanying drawings and the description below. Furtherfeatures, aspects, and advantages will become apparent from thedescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A-C show audio consumers in a range of consumption modalities,including using headphones fed information from a mobile device (1A), ina car or other form of transportation (1B), and in an interior space(1C).

FIG. 2 shows headphones including a haptic device.

FIG. 3 shows a block diagram of an audio reproduction system.

FIG. 4 shows a block diagram of a device capable of playing audio files.

FIG. 5 shows steps for processing information for reproduction in areproduction device.

FIG. 6 shows steps for obtaining and applying sound profiles.

FIG. 7 shows an exemplary user interface by which the user can inputgeographic, consumption modality, and demographic information for use insound profiles.

FIG. 8 shows an exemplary user interface by which the user can determinewhich aspects of tuning should be utilized in applying a sound profile.

FIGS. 9A-B show subscreens of an exemplary user interface by which theuser has made detailed changes to the dynamic equalization settings ofsound profiles for songs in two different genres.

FIG. 10 shows an exemplary user interface by which the user can sharethe sound profile settings the user or the user's contacts have chosen.

FIG. 11 shows steps undertaken by a computer with a sound profiledatabase receiving a sound profile request.

FIG. 12 shows steps undertaken by a computer with a sound profiledatabase receiving a user-modified sound profile.

FIG. 13 shows a block diagram of a computer system capable ofmaintaining sound profile database and providing sound profiles tousers.

FIG. 14 shows how a computer system can provide sound profiles tomultiple users.

FIG. 15 shows steps undertaken by a computer to analyze a user's musiccollection to allow for intensity-based content selection.

FIGS. 16A-B show an exemplary user interface by which the user canperform intensity-based content selection.

FIG. 17 shows exemplary steps performed by a device to determineparameters for a sound profile based on a user's sensitivity acrossvarious frequency ranges.

FIGS. 18A-C show an exemplary audible frequency region divided intodifferent frequency ranges, some of which may be inaudible to individualusers.

FIG. 19 shows exemplary steps performed by a device to detect aninaudible frequency range and to generate a lowest volume in an audiblefrequency range for individual users.

FIG. 20 shows exemplary steps performed by a device to detect hapticsensitivity across various frequency ranges.

FIGS. 21A-C show a user interface for detecting a user's sensitivity tosounds or haptic movement across various frequency ranges.

FIGS. 22A-B shows an exemplary haptic device and its components.

FIG. 23 shows exemplary steps performed by a device to predict and use asound profile.

Like reference symbols indicate like elements throughout thespecification and drawings.

DETAILED DESCRIPTION

In FIG. 1A, the user 105 is using headphones 120 in a consumptionmodality 100. Headphones 120 can be of the on-the-ear or over-the-eartype. Headphones 120 can be connected to mobile device 110. Mobiledevice 110 can be a smartphone, portable music player, portable videogame or any other type of mobile device capable of generatingentertainment by reproducing audio files. In some implementations,mobile device 110 can be connected to headphone 120 using audio cable130, which allows mobile device 110 to transmit an audio signal toheadphones 120. Such cable 130 can be a traditional audio cable thatconnects to mobile device 110 using a standard headphone jack. The audiosignal transmitted over cable 130 can be of sufficient power to drive,i.e., create sound, at headphones 120. In other implementations, mobiledevice 110 can alternatively connect to headphones 120 using wirelessconnection 160. Wireless connection 160 can be a Bluetooth, Low PowerBluetooth, or other networking connection. Wireless connection 160 cantransmit audio information in a compressed or uncompressed format. Theheadphones would then provide their own power source to amplify theaudio data and drive the headphones. Mobile device 110 can connect toInternet 140 over networking connection 150 to obtain the sound profile.Networking connection 150 can be wired or wireless.

Headphones 120 can include stereo speakers including separate driversfor the left and right ear to provide distinct audio to each ear.Headphones 120 can include a haptic device 170 to create a basssensation by providing vibrations through the top of the headphone band.Headphone 120 can also provide vibrations through the left and right earcups using the same or other haptic devices. Headphone 120 can includeadditional circuitry to process audio and drive the haptic device.

Mobile device 110 can play compressed audio files, such as those encodedin MP3 or AAC format. Mobile device 110 can decode, obtain, and/orrecognize metadata for the audio it is playing back, such as through ID3tags or other metadata. The audio metadata can include the name of theartists performing the music, the genre, and/or the song title. Mobiledevice 110 can use the metadata to match a particular song, artist, orgenre to a predefined sound profile. The predefined sound profile can beprovided by Alpine and downloaded with an application or retrieved fromthe cloud over networking connection 150. If the audio does not havemetadata (e.g., streaming situations), a sample of the audio can be sentand used to determine the genre and other metadata.

Such a sound profile can include which frequencies or audio componentsto enhance or suppress, e.g., through equalization, signal processing,and/or dynamic noise reduction, allowing the alteration of thereproduction in a way that enhances the auditory experience. The soundprofiles can be different for the left and right channel. For example,if a user requires a louder sound in one ear, the sound profile canamplify that channel more. Other known techniques can also be used tocreate three-dimensional audio effects. In another example, theimmersion experience can be tailored to specific music genres. Forexample, with its typically narrower range of frequencies, the easylistening genre may benefit from dynamic noise compression, whilebass-heavy genres (i.e., hip-hop, dance music, and rap) can haveenhanced bass and haptic output. Although the immersive initial settingsare a unique blending of haptic, audio, and headphone clamping forces,the end user can tune each of these aspects (e.g., haptic, equalization,signal processing, dynamic noise reduction, 3D effects) to suit his orher tastes. Genre-based sound profiles can include rock, pop, classical,hip-hop/rap, and dance music. In another implementation, the soundprofile could modify the settings for Alpine's MX algorithm, aproprietary sound enhancement algorithm, or other sound enhancementalgorithms known in the art.

Mobile device 110 can obtain the sound profiles in real time, such aswhen mobile device 110 is streaming music, or can download soundprofiles in advance for any music or audio stored on mobile device 110.As described in more detail below, mobile device 110 can allow users totune the sound profile of their headphone to their own preferencesand/or apply predefined sound profiles suited to the genre, artist,song, or the user. For example, mobile device 110 can use Alpine'sTune-It mobile application. Tune-It can allow users quickly modify theirheadphone devices to suite their individual tastes. Additionally,Tune-It can communicate settings and parameters (metadata) to a serveron the Internet, and allow the server to associate sound settings withmusic genres.

Audio cable 130 or wireless connection 160 can also transmit non-audioinformation to or from headphones 120. The non-audio informationtransmitted to headphones 120 can include sound profiles. The non-audioinformation transmitted from headphones 120 may include deviceinformation, e.g., information about the headphones themselves,geographic or demographic information about user 105. Such deviceinformation can be used by mobile device 110 in its selection of a soundprofile, or combined with additional device information regarding mobiledevice 110 for transmission over the Internet 140 to assist in theselection of a sound profile in the cloud.

Given their proximity to the ears, when headphones 120 are used toexperience auditory entertainment, there is often less interferencestemming from the consumption modality itself beyond ambient noise.Other consumption modalities present challenges to the auditoryexperience, however. For example, FIG. 1B depicts the user in adifferent modality, namely inside an automobile or analogous mode oftransportation such as car 101. Car 101 can have a head unit 111 thatplays audio from AM broadcasts, FM broadcasts, CDs, DVDs, flash memory(e.g., USB thumb drives), a connected iPod or iPhone, mobile device 110,or other devices capable of storing or providing audio. Car 101 can havefront left speakers 182, front right speakers 184, rear left speakers186, and rear right speakers 188. Head unit 111 can separately controlthe content and volume of audio sent to speakers 182, 184, 186, and 188.Car 101 can also include haptic devices for each seat, including frontleft haptic device 183, front right haptic device 185, rear left hapticdevice 187, and rear right haptic device 189. Head unit 111 canseparately control the content and volume reproduced by haptic devices183, 185, 187, and 189.

Head unit 111 can create a single low frequency mono channel that driveshaptic devices 183, 185, 187, and 189, or head unit 111 can separatelydrive each haptic device based off the audio sent to the adjacentspeaker. For example, haptic device 183 can be driven based on thelow-frequency audio sent to speaker 182. Similarly, haptic devices 185,187, and 189 can be driven based on the low-frequency audio sent tospeakers 184, 186, and 188, respectively. Each haptic device can beoptimized for low, mid, and high frequencies.

Head unit 111 can utilize sound profiles to optimize the blend of audioand haptic sensation. Head unit 111 can use sound profiles as they aredescribed in reference to mobile device 110 and headset 200.

While some modes of transportation are configured to allow a mobiledevice 110 to provide auditory entertainment directly, some have a headunit 111 that can independently send information to Internet 140 andreceive sound profiles, and still others have a head unit that cancommunicate with a mobile device 110, for example by Bluetoothconnection 112. Whatever the specific arrangement, a networkingconnection 150 can be made to the Internet 140, over which audio data,associated metadata, and device information can be transmitted as wellas sound profiles can be obtained.

In such a transportation modality, there may be significant ambientnoise that must be overcome. Given the history of car stereos, manyusers in the transportation modality have come to expect a bass-heavysound for audio played in a transportation modality. Reflection andabsorbance of sound waves by different materials in the passenger cabinmay impact the sounds perceived by passengers, necessitatingequalization and compensations. Speakers located in different placeswithin the passenger cabin, such as a front speaker 182 and a rearspeaker 188 may generate sound waves that reach passengers at differenttimes, necessitating the introduction of a time delay so each passengerreceives the correct compilation of sound waves at the correct moment.All of these modifications to the audio reproduction—as well as othersbased on the user's unique preferences or suited to the genre, artist,song, the user, or the reproduction device—can be applied either byhaving the user tune the sound profile or by applying predefined soundprofiles.

Another environment in which audio entertainment is routinelyexperienced is modality 102, an indoor modality such as the one depictedin FIG. 1C as a room inside a house. In such an indoor modality, theaudio entertainment may come from a number of devices, such as mobiledevice 110, television 113, media player 114, stereo 115, videogamesystem 116, or some combination thereof wherein at least one of thedevices is connected to Internet 140 through networking connection 150.In modality 102, user 105 may choose to experience auditoryentertainment through wired or wireless headphones 120, or via speakersmounted throughout the interior of the space. The speakers could bestereo speakers or surround sound speakers. As in modality 101, inmodality 102 reflection and absorbance of sound waves and speakerplacement may necessitate modification of the audio data to enhance theauditory experience. Other effects may also be desirable and enhance theaudio experience in such an environment. For example, if a user isutilizing headphones in close proximity to someone who is not, dynamicnoise compression may help the user from disturbing the nonuser. Suchmodifications—as well as others based on the user's unique preferences,demographics, or geography, the reproduction device, or suited to thegenre, artist, song, or the user—can be applied either by having theuser tune the sound profile in modality 102 or by applying predefinedsound profiles during reproduction in modality 102.

Similarly, audio entertainment could be experienced outdoors on a patioor deck, in which case there may be almost no reflections. In additionto the various criteria described above, device information includingdevice identifiers or location information could be used toautomatically identify an outdoor consumption modality, or a user couldmanually input the modality. As in the other modalities, sound profilescan be used to modify the audio data so that the auditory experience isenhanced and optimized.

With more users storing and/or accessing media remotely, users willexpect their preferences for audio reproduction to be carried acrossdifferent modalities, such as those represented in FIGS. 1A-C. Forexample, if a user makes a change in the sound profile for a song whileexperiencing it in modality 101, the user may expect that same changewill be present when next listening to the same song in modality 102.Given the different challenges inherent in each of the consumptionmodalities, however, not to mention the different reproduction devicesthat may be present in each modality, for the audio experience to beenhanced and optimized, such user-initiated changes in one modality mayneed to be harmonized or combined with other, additional modificationsunique to the second modality. These multiple and complex modificationscan be accomplished through sound profiles, even if the user does notnecessarily appreciate the intricacies involved.

FIG. 2 shows headphones including a haptic device. In particular,headphones 200 includes headband 210. Right ear cup 220 is attached toone end of headband 210. Right ear cup 220 can include a driver thatpushes a speaker to reproduce audio. Left ear cup 230 is attached to theopposite end of headband 210 and can similarly include a driver thatpushes a speaker to reproduce audio. The top of headband 210 can includehaptic device 240. Haptic device 240 can be covered by cover 250.Padding 245 can cover the cover 250. Right ear cup 220 can include apower source 270 and recharging jack 295. Left ear cup 230 can includesignal processing components 260 inside of it, and headphone jack 280.Left ear cup 230 can have control 290 attached. Headphone jack 280 canaccept an audio cable to receive audio signals from a mobile device.Control 290 can be used to adjust audio settings, such as to increasethe bass response or the haptic response. In other implementations, thelocation of power source 270, recharging jack 295, headphone jack 280,and signal processing components 260 can swap ear cups, or be combinedinto either single ear cup.

Multiple components are involved in both the haptic and sound profilefunctions of the headphones. These functions are discussed on acomponent-by-component basis below.

Power source 270 can be a battery or other power storage device known inthe art. In one implementation it can be one or more batteries that areremovable and replaceable. For example, it could be an AAA alkalinebattery. In another implementation it could be a rechargeable batterythat is not removable. Right ear cup 270 can include recharging jack 295to recharge the battery. Recharging jack 295 can be in the micro USBformat. Power source 270 can provide power to signal processingcomponents 260. Power source 270 can provide power to signal processingcomponents 260. Power source 270 can last at least 10 hours.

Signal processing components 260 can receive stereo signals fromheadphone jack 280 or through a wireless networking device, processsound profiles received from headphone jack 280 or through wirelessnetworking, create a mono signal for haptic device 240, and amplify themono signal to drive haptic device 240. In another implementation,signal processing components 260 can also amplify the right audiochannel that drives the driver in the right ear cup and amplify the leftaudio channel that drives the left audio cup. Signal processingcomponents 260 can deliver a low pass filtered signal to the hapticdevice that is mono in nature but derived from both channels of thestereo audio signal. Because it can be difficult for users todistinguish the direction or the source of bass in a home or automotiveenvironment, combining the low frequency signals into a mono signal forbass reproduction can simulate a home or car audio environment. Inanother implementation, signal processing components 260 can deliverstereo low-pass filtered signals to haptic device 240.

In one implementation, signal processing components 260 can include ananalog low-pass filter. The analog low-pass filter can use inductors,resistors, and/or capacitors to attenuate high-frequency signals fromthe audio. Signal processing components 260 can use analog components tocombine the signals from the left and right channels to create a monosignal, and to amplify the low-pass signal sent to haptic device 240.

In another implementation, signal processing components 260 can bedigital. The digital components can receive the audio information, via anetwork. Alternatively, they can receive the audio information from ananalog source, convert the audio to digital, low-pass filter the audiousing a digital signal processor, and provide the low-pass filteredaudio to a digital amplifier.

Control 290 can be used to modify the audio experience. In oneimplementation, control 290 can be used to adjust the volume. In anotherimplementation, control 290 can be used to adjust the bass response orto separately adjust the haptic response. Control 290 can provide aninput to signal processing components 260.

Haptic device 240 can be made from a small transducer (e.g., a motorelement) which transmits low frequencies (e.g., 1 Hz-100 Hz) to theheadband. The small transducer can be less than 1.5″ in size and canconsume less than 1 watt of power. Haptic device 240 can be an off-theshelf haptic device commonly used in touch screens or for exciters toturn glass or plastic into a speaker. Haptic device 240 can use a voicecoil or magnet to create the vibrations.

Haptic device 240 can be positioned so it is displacing directly on theheadband 210. This position allows much smaller and thus power efficienttransducers to be utilized. The housing assembly for haptic device 240,including cover 250, is free-floating, which can maximize articulationof haptic device 240 and reduces dampening of its signal.

The weight of haptic device 240 can be selected as a ratio to the massof the headband 210. The mass of haptic device 240 can be selecteddirectly proportional to the rigid structure to enable sufficientacoustic and mechanical energy to be transmitted to the ear cups. If themass of haptic device 240 were selected to be significantly lower thanthe mass of the headband 210, then headband 210 would dampen allmechanical and acoustic energy. Conversely, if the mass of haptic device240 were significantly higher than the mass of the rigid structure, thenthe weight of the headphone would be unpleasant for extended usage andmay lead to user fatigue. Haptic device 240 is optimally placed in thetop of headband 210. This positioning allows the gravity of the headbandto generate a downward force that increases the transmission ofmechanical vibrations from the haptic device to the user. The top of thehead also contains a thinner layer of skin and thus locating hapticdevice 240 here provides more proximate contact to the skull. The uniqueposition of haptic device 240 can enable the user to experience animmersive experience that is not typically delivered via traditionalheadphones with drivers located merely in the headphone cups.

The haptic device can limit its reproduction to low frequency audiocontent. For example, the audio content can be limited to less than 100Hz. Vibrations from haptic device 240 can be transmitted from hapticdevice 240 to the user through three contact points: the top of theskull, the left ear cup, and the right ear cup. This creates animmersive bass experience. Because headphones have limited power storagecapacities and thus require higher energy efficiencies to satisfydesired battery life, the use of a single transducer in a location thatmaximizes transmission across the three contact points also creates apower-efficient bass reproduction.

Cover 250 can allow haptic device 240 to vibrate freely. Headphone 200can function without cover 250, but the absence of cover 250 can reducethe intensity of vibrations from haptic device 240 when a user's skullpresses too tightly against haptic device 240.

Padding 245 covers haptic device 240 and cover 250. Depending on itssize, shape, and composition, padding 245 can further facilitate thetransmission of the audio and mechanical energy from haptic device 240to the skull of a user. For example, padding 245 can distribute thetransmission of audio and mechanical energy across the skull based onits size and shape to increase the immersive audio experience. Padding245 can also dampen the vibrations from haptic device 240.

Headband 210 can be a rigid structure, allowing the low frequency energyfrom haptic device 240 to transfer down the band, through the left earcup 230 and right ear cup 220 to the user. Forming headband 210 of arigid material facilitates efficient transmission of low frequency audioto ear cups 230 and 220. For example, headband 210 can be made from hardplastic like polycarbonate or a lightweight metal like aluminum. Inanother implementation, headband 210 can be made from spring steel.Headband 210 can be made such that the material is optimized formechanical and acoustic transmissibility through the material. Headband210 can be made by selecting specific type materials as well as a formfactor that maximizes transmission. For example, by utilizing reinforcedribbing in headband 210, the amount of energy dampened by the rigid bandcan be reduced and enable more efficient transmission of the mechanicaland acoustic frequencies to be passed to the ear cups 220 and 230.

Headband 210 can be made with a clamping force measured between ear cups220 and 230 such that the clamping force is not so tight as to reducevibrations and not so loose as to minimize transmission of thevibrations. The clamping force can be in the range of 300 g to 700 g.

Ear cups 220 and 230 can be designed to fit over the ears and to coverthe whole ear. Ear cups 220 and 230 can be designed to couple andtransmit the low frequency audio and mechanical energy to the user'shead. Ear cups 220 and 230 may be static. In another implementation, earcups 220 and 230 can swivel, with the cups continuing to be attached toheadband 210 such that they transmit audio and mechanical energy fromheadband 210 to the user regardless of their positioning.

Vibration and audio can be transmitted to the user via multiple methodsincluding auditory via the ear canal, and bone conduction via the skullof the user. Transmission via bone conduction can occur at the top ofthe skull and around the ears through ear cups 220 and 230. This featurecreates both an aural and tactile experience for the user that issimilar to the audio a user experiences when listening to audio from asystem that uses a subwoofer. For example, this arrangement can create aheadphone environment where the user truly feels the bass.

In another aspect, some or all of the internal components could be foundin an amplifier and speaker system found in a house or a car. Forexample, the internal components of headphone 200 could be found in acar stereo head unit with the speakers found in the dash and doors ofthe car.

FIG. 3 shows a block diagram of a reproduction system 300 that can beused to implement the techniques described herein for an enhanced audioexperience. Reproduction system 300 can be implemented inside ofheadphones 200. Reproduction system 300 can be part of signal processingcomponents 260. Reproduction system 300 can include bus 365 thatconnects the various components. Bus 365 can be composed of multiplechannels or wires, and can include one or more physical connections topermit unidirectional or omnidirectional communication between two ormore of the components in reproduction system 300. Alternatively,components connected to bus 365 can be connected to reproduction system300 through wireless technologies such as Bluetooth, Wifi, or cellulartechnology.

An input 340 including one or more input devices can be configured toreceive instructions and information. For example, in someimplementations input 340 can include a number of buttons. In some otherimplementations input 340 can include one or more of a touch pad, atouch screen, a cable interface, and any other such input devices knownin the art. Input 340 can include knob 290. Further, audio and imagesignals also can be received by the reproduction system 300 through theinput 340.

Headphone jack 310 can be configured to receive audio and/or datainformation. Audio information can include stereo or other multichannelinformation. Data information can include metadata or sound profiles.Data information can be sent between segments of audio information, forexample between songs, or modulated to inaudible frequencies andtransmitted with the audio information.

Further, reproduction system 300 can also include network interface 380.Network interface 380 can be wired or wireless. A wireless networkinterface 380 can include one or more radios for making one or moresimultaneous communication connections (e.g., wireless, Bluetooth, lowpower Bluetooth, cellular systems, PCS systems, or satellitecommunications). Network interface 380 can receive audio information,including stereo or multichannel audio, or data information, includingmetadata or sound profiles.

An audio signal, user input, metadata, other input or any portion orcombination thereof can be processed in reproduction system 300 usingthe processor 350. Processor 350 can be used to perform analysis,processing, editing, playback functions, or to combine various signals,including adding metadata to either or both of audio and image signals.Processor 350 can use memory 360 to aid in the processing of varioussignals, e.g., by storing intermediate results. Processor 350 caninclude A/D processors to convert analog audio information to digitalinformation. Processor 350 can also include interfaces to pass digitalaudio information to amplifier 320. Processor 350 can process the audioinformation to apply sound profiles, create a mono signal and apply lowpass filter. Processor 350 can also apply Alpine's MX algorithm.

Processor 350 can low pass filter audio information using an active lowpass filter to allow for higher performance and the least amount ofsignal attenuation. The low pass filter can have a cut off ofapproximately 80 Hz-100 Hz. The cut off frequency can be adjusted basedon settings received from input 340 or network 380. Processor 350 canparse and/or analyze metadata and request sound profiles via network380.

In another implementation, passive filter 325 can combine the stereoaudio signals into a mono signal, apply the low pass filter, and sendthe mono low pass filter signal to amplifier 320.

Memory 360 can be volatile or non-volatile memory. Either or both oforiginal and processed signals can be stored in memory 360 forprocessing or stored in storage 370 for persistent storage. Further,storage 370 can be integrated or removable storage such as SecureDigital, Secure Digital High Capacity, Memory Stick, USB memory, compactflash, xD Picture Card, or a hard drive.

The audio signals accessible in reproduction system 300 can be sent toamplifier 320. Amplifier 320 can separately amplify each stereo channeland the low-pass mono channel. Amplifier 320 can transmit the amplifiedsignals to speakers 390 and haptic device 240. In anotherimplementation, amplifier 320 can solely power haptic device 240.Amplifier 320 can consume less than 2.5 Watts.

While reproduction system 300 is depicted as internal to a pair ofheadphones 200, it can also be incorporated into a home audio system ora car stereo system.

FIG. 4 shows a block diagram of mobile device 110, head unit 111, stereo115 or other device similarly capable of playing audio files. FIG. 4presents a computer system 400 that can be used to implement thetechniques described herein for sharing digital media. Computer system400 can be implemented inside of mobile device 110, head unit 111,stereo 115, or other device similar capable of playing audio files. Bus465 can include one or more physical connections and can permitunidirectional or omnidirectional communication between two or more ofthe components in the computer system 400. Alternatively, componentsconnected to bus 465 can be connected to computer system 400 throughwireless technologies such as Bluetooth, Wifi, or cellular technology.The computer system 400 can include a microphone 445 for receiving soundand converting it to a digital audio signal. The microphone 445 can becoupled to bus 465, which can transfer the audio signal to one or moreother components. Computer system 400 can include a headphone jack 460for transmitting audio and data information to headphones and otheraudio devices.

An input 440 including one or more input devices also can be configuredto receive instructions and information. For example, in someimplementations input 440 can include a number of buttons. In some otherimplementations input 440 can include one or more of a mouse, akeyboard, a touch pad, a touch screen, a joystick, a cable interface,voice recognition, and any other such input devices known in the art.Further, audio and image signals also can be received by the computersystem 400 through the input 440 and/or microphone 445.

Further, computer system 400 can include network interface 420. Networkinterface 420 can be wired or wireless. A wireless network interface 420can include one or more radios for making one or more simultaneouscommunication connections (e.g., wireless, Bluetooth, low powerBluetooth, cellular systems, PCS systems, or satellite communications).A wired network interface 420 can be implemented using an Ethernetadapter or other wired infrastructure.

Computer system 400 may include a GPS receiver 470 to determine itsgeographic location. Alternatively, geographic location information canbe programmed into memory 415 using input 440 or received via networkinterface 420. Information about the consumption modality, e.g., whetherit is indoors, outdoors, etc., may similarly be retrieved or programmed.The user may also personalize computer system 400 by indicating theirage, demographics, and other information that can be used to tune soundprofiles.

An audio signal, image signal, user input, metadata, geographicinformation, user, reproduction device, or modality information, otherinput or any portion or combination thereof, can be processed in thecomputer system 400 using the processor 410. Processor 410 can be usedto perform analysis, processing, editing, playback functions, or tocombine various signals, including parsing metadata to either or both ofaudio and image signals.

For example, processor 410 can parse and/or analyze metadata from a songor video stored on computer system 400 or being streamed across networkinterface 420. Processor 410 can use the metadata to request soundprofiles from the Internet through network interface 420 or from storage430 for the specific song, game or video based on the artist, genre, orspecific song or video. Processor 410 can provide information throughthe network interface 420 to allow selection of a sound profile based ondevice information such as geography, user ID, user demographics, deviceID, consumption modality, the type of reproduction device (e.g., mobiledevice, head unit, or Bluetooth speakers), reproduction device, orspeaker arrangement (e.g., headphones plugged or multi-channel surroundsound). The user ID can be anonymous but specific to an individual useror use real world identification information.

Processor 410 can then use input received from input 440 to modify asound profile according to a user's preferences. Processor 410 can thentransmit the sound profile to a headphone connected through networkinterface 420 or headphone jack 460 and/or store a new sound profile instorage 430. Processor 410 can run applications on computer system 400like Alpine's Tune-It mobile application, which can adjust soundprofiles. The sound profiles can be used to adjust Alpine's MXalgorithm.

Processor 410 can use memory 415 to aid in the processing of varioussignals, e.g., by storing intermediate results. Memory 415 can bevolatile or non-volatile memory. Either or both of original andprocessed signals can be stored in memory 415 for processing or storedin storage 430 for persistent storage. Further, storage 430 can beintegrated or removable storage such as Secure Digital, Secure DigitalHigh Capacity, Memory Stick, USB memory, compact flash, xD Picture Card,or a hard drive.

Image signals accessible in computer system 400 can be presented on adisplay device 435, which can be an LCD display, printer, projector,plasma display, or other display device. Display 435 also can displayone or more user interfaces such as an input interface. The audiosignals available in computer system 400 also can be presented throughoutput 450. Output device 450 can be a speaker, multiple speakers,and/or speakers in combination with one or more haptic devices.Headphone jack 460 can also be used to communicate digital or analoginformation, including audio and sound profiles.

Computer system 400 could include passive filter 325, amplifier 320,speaker 390, and haptic device 240 as describe above with reference toFIG. 3, and be installed inside headphone 200.

FIG. 5 shows steps for processing information for reproduction inheadphones or other audio reproduction devices. Headphones can monitor aconnection to determine when audio is received, either through an analogconnection or digitally (505). When audio is received, any analog audiocan be converted from analog to digital (510) if a digital filter isused. The sound profile can be adjusted according to user input (e.g., acontrol knob) on the headphones (515). The headphones can apply a soundprofile (520). The headphones can then create a mono signal (525) usingknown mixing techniques. The mono signal can be low-pass filtered (530).The low-pass filtered mono signal can be amplified (535). In someimplementations (e.g., when the audio is digital), the stereo audiosignal can also be amplified (540). The amplified signals can then betransmitted to their respective drivers (545). For example, the low-passfiltered mono signal can be sent to a haptic device and the amplifiedleft and right channel can be sent to the left and right driversrespectively.

FIGS. 3 and 4 show systems capable of performing these steps. The stepsdescribed in FIG. 5 need not be performed in the order recited and twoor more steps can be performed in parallel or combined. In someimplementations, other types of media also can be shared or manipulated,including audio or video.

FIG. 6 shows steps for obtaining and applying sound profiles. Mobiledevice 110, head unit 111, stereo 115 or other device similarly capableof playing audio files can wait for media to be selected forreproduction or loaded onto a mobile device (605). The media can be asong, album, game, or movie. Once the media is selected, metadata forthe media is parsed and/or analyzed to determine if the media containsmusic, voice, or a movie, and what additional details are available suchas the artist, genre or song name (610). Additional device information,such as geography, user ID, user demographics, device ID, consumptionmodality, the type of reproduction device (e.g., mobile device, headunit, or Bluetooth speakers), reproduction device, or speakerarrangement (e.g., headphones plugged or multi-channel surround sound),may also be parsed and/or analyzed in step 610. The parsed/analyzed datais used to request a sound profile from a server over a network, such asthe Internet, or from local storage (615). For example, Alpine couldmaintain a database of sound profiles matched to various types of mediaand matched to various types of reproduction devices. The sound profilecould contain parameters for increasing or decreasing various frequencybands and other sound parameters for enhancing portions of the audio.Such aspects could include dynamic equalization, crossover gain, dynamicnoise compression, time delays, and/or three-dimensional audio effects.Alternatively, the sound profile could contain parameters for modifyingAlpine's MX algorithm. The sound profile is received (620) and thenadjusted to a particular user's preference (625) if necessary. Theadjusted sound profile is then transmitted (630) to a reproductiondevice, such as a pair of headphones. The adjusted profile and itsassociated metadata can also be transmitted (640) to the server wherethe sound profile, its metadata, and the association is stored, both forlater analysis and use by the user.

FIGS. 3 and 4 show systems capable of performing these steps. The stepsdescribed in FIG. 6 could also be performed in headphones connected to anetwork without the need of an additional mobile device. The stepsdescribed in FIG. 6 need not be performed in the order recited and twoor more steps can be performed in parallel or combined. In someimplementations, other types of media also can be shared or manipulated,including audio or video.

FIG. 7 shows an exemplary user interface by which the user can inputgeographic, consumption modality, and demographic information for use increating or retrieving sound profiles for a reproduction device such asmobile device 110, head unit 111, or stereo 115. Field 710 allows theuser to input geographical information in at least two ways. First,switch 711 allows the user to activate or deactivate the GPS receiver.When activated, the GPS receiver can identify the current geographicalposition of device 110, and uses that location as the geographicalparameter when selecting a sound profile. Alternatively, the user canset a geographical preference using some sort of choosing mechanism,such as the drop-down list 712. Given the wide variety of effectivetechniques for creating user interfaces, one skilled in the art willalso appreciate many alternative mechanisms by which such geographicselection could be accomplished. Field 720 of the user interfacedepicted in FIG. 7 allows the user to select among various modalities inwhich the user may be experiencing the audio entertainment. Whiledrop-down list 721 is one potential tool for this task, one skilled inthe art will appreciate that others could be equally effective. Theuser's selection in field 720 can be used as the modality parameter whenselecting a sound profile. Field 730 of the user interface depicted inFIG. 7 allows the user to input certain demographic information for usein selecting a sound profile. One such piece of information could beage, given the changing musical styles and preferences among differentgenerations. Similarly, ethnicity and cultural information could be usedas inputs to account for varying musical preferences within the countryand around the world. This information can also be inferred based onmetadata patterns found in media preferences. Again, drop-down 731 isshown as one potential tool for this task, while other, alternativetools could also be used.

FIG. 8 shows an exemplary user interface by which the user can selectwhich aspects of tuning should be utilized when a sound profile isapplied. Field 810 corresponds to dynamic equalization, which can beactivated or deactivated by a switch such as item 811. When dynamicequalization is activated, selector 812 allows the user to select whichtype of audio entertainment the user wishes to manually adjust, whileselector 813 presents subchoices within each type. For example, if auser selects “Music” with selector 812, selector 813 could presentdifferent genres, such as “Rock,” “Jazz,” and “Classical.” Based on theuser's choice, a genre-specific sound profile can be retrieved frommemory or the server, and either used as-is or further modified by theuser using additional interface elements on subscreens that can appearwhen dynamic equalization is activated. Fields 820, 830, and 840 operatein similar fashion, allowing the user to activate or deactivate tuningaspects such as noise compression, crossover gain, and advanced featuresusing switches 821, 831, 831, and 842. As each aspect is activated,controls specific to each aspect can be revealed to the user. Forexample, turning on noise compression can reveal a sider that controlsthe amount of noise compression. Turning on crossover gain can revealsliders that control both crossover frequency and one or more gains.While the switches presented represent one interface tool for activatingand deactivating these aspects, one will appreciate that other,alternative interface tools could be employed to achieve similarresults.

FIGS. 9A-B show subscreens of an exemplary user interface by which theuser can make detailed changes to the equalization settings of soundprofiles for songs in two different genres, one “Classical” and one “HipHop.” Similarly to the structures discussed with respect to FIG. 8,selector 910 allows the user to select which type of audio entertainmentthe user can be experiencing, while selector 920 provides choices withineach type. Here, because “Music” has been selected with selector 910,musical genres are represented on selector 920. In FIG. 9A, the user hasselected the “Classical” genre, and therefore the predefined soundprofile for dynamic equalization for the “Classical” genre has beenloaded. Five frequency bands are presented as vertical ranges 930. Morefrequency bands are possible. Each range is equipped with a slider 940that begins at the value predefined for that range in “Classical” music.The user can manipulate any or all of these sliders up or down alongtheir vertical ranges 930 to modify the sound presented. In field 950,the level of “Bass” begins where it is preset for “Classical” music,i.e., the “low” value, but the selector can be used to adjust the levelof “Bass” to “High” or “Off.” In another aspect, an additional field for“Bass sensation” that maps to haptic feedback can be presented. In FIG.9B, the user has selected a different genre of Music, i.e., “Hip Hop.”Accordingly, all of the dynamic equalization and Bass settings are thepredefined values for the “Hip Hop” sound profile, and one can see thatthese are different than the values for “Classical.” As in FIG. 9A, ifthe user wishes, the user can modify any or all of the settings in FIG.9B. As one skilled in the art will appreciate, the controls of theinterface presented in FIGS. 9A and 9B could be accomplished withalternative tools. Similarly, although similar subscreens have not beenpresented for each of the other aspects of tuning, similar subscreenswith additional controls can be utilized for crossover gain, dynamicnoise compression, time delays, and/or three-dimensional audio effects.

FIG. 10 shows an exemplary user interface by which the user can sharethe sound profile settings the user or the user's contacts have chosen.User's identification is represented by some sort of user identification1010, whether that is an actual name, a screen name, or some other kindof alias. The user can also be represented graphically, by some kind ofpicture or avatar 1011. The user interface in FIG. 10 contains an“Activity” region 1020 that can update periodically but which can bemanually updated using a control such as refresh button 1021. Within“Activity” region 1020, a number of events 1030 are displayed. Eachevent 1030 contains detail regarding the audio file experienced byanother user 1031—again identified by some kind of moniker, picture, oravatar—and which sound profile 1032 was used to modify it. In FIG. 10,the audio file being listened to during each event 1030 is representedby an album cover 1033, but could be represented in other ways. The userinterface allows the user to choose to experience the same audio filelistened to by the other user 1031 by selecting it from activity region1030. The user is then free to use the same sound profile 1032 as theother user 1031, or to decide for him or herself how the audio should betuned according to the techniques described earlier herein.

In addition to following the particular audio events of certain otherusers in the “Activity” region 1020, the user interface depicted in FIG.10 contains a “Suggestion” region 1040. Within “Suggestion” region 1040,the user interface is capable of making suggestions of additional usersto follow, such as other user 1041, based on their personal connectionsto the user, their personal connection to those other users beingfollowed by the user, or having similar audio tastes to the user basedon their listening preferences or history 1042.

FIGS. 3 and 4 show systems capable of providing the user interfacediscuss in FIGS. 7-10.

FIG. 11 shows steps undertaken by a computer with a sound profiledatabase receiving a sound profile request. The computer can be a localcomputer or stored in the cloud, on a server on a network, including theInternet. In particular, the database, which is connected to a networkfor communication, may receive a sound profile request (1105) fromdevices such as mobile device 110 referred to above. Such a request canprovide device information and audio metadata identifying what kind ofsound profile is being requested, and which user is requesting it. Inanother aspect, the request can contain an audio sample, which can beused to identify the metadata. Accordingly, the database is able toidentify the user making the request (1110) and then search storage forany previously-modified sound profiles created and stored by the userthat match the request (1115). If such a previously-modified profilematching the request exists in storage, the database is able to transmitit to the user over a network (1120). If no such previously-modifiedprofile matching the request exists, the database works to analyze dataincluded in the request to determine what preexisting sound profilesmight be suitable (1125). For example, as discussed elsewhere herein,basic sound profiles could be archived in the database corresponding todifferent metadata such as genres of music, the artist, or song name.Similarly, the database could be loaded with sound profilescorresponding to specific reproduction devices or basic consumptionmodalities. The user may have identified his or her preferred geography,either as a predefined location or by way of the GPS receiver in theuser's audio reproduction device. That information may allow for themodification of the generic genre profile in light of certain geographicreproduction preferences. Similar analysis and extrapolation may beconducted on the basis of demographic information, the specificconsumption modality (e.g., indoors, outdoors, in a car, etc),reproduction devices, and so forth. As discussed in more detail below,if audio files are assigned certain intensity scores, sound profilescould be associated with intensity levels so that a user can make arequest based on the intensity of music the user wishes to hear. Asanother example, the database may have a sound profile for a similarreproduction device, for the same song, created by someone on the samestreet, which suggests that sound profile would be a good match. Theweighting of the different criteria in selecting a “best match” soundprofile can vary. For example the reproduction device may carry greaterweight than the geography. Once the data is analyzed and a suitablesound profile is identified and/or modified based on the data, the soundprofile is transmitted over a network to the user (1130). Such adatabase could be maintained as part of a music streaming service, orother store that sells audio entertainment.

For example, the computer or set of computers could also maintaining alibrary of audio or media files for download or streaming by users. Theaudio and media files would have metadata, which could include intensityscores. When a user or recommendation engine selects media for downloador streaming, the metadata for that media could be used to transmit auser's stored, modified sound profile (1120) or whatever preexistingsound profile might be suitable (1125). The computer can then transmitthe sound profile with the media or transmit it or transmit it lessfrequency if the sound profile is suitable for multiple pieces ofsubsequent media (e.g. if a user selects a genre on a streaming station,the computer system may only need to send a sound profile for the firstsong of that genre, at least until the user switches genres).

Computer system 400 and computer system 1300 show systems capable ofperforming these steps. A subset of components in computer system 400 orcomputer system 1300 could also be used, and the components could befound in a PC, server, or cloud-based system. The steps described inFIG. 11 need not be performed in the order recited and two or more stepscan be performed in parallel or combined.

FIG. 12 shows steps undertaken by a computer with a sound profiledatabase receiving a user-modified sound profile. In particular, once auser modifies an existing sound profile as previously described herein,the user's audio reproduction device can transmit the modified soundprofile over a network back to the database at the first convenientopportunity. The modified sound profile is received at the database(1205), and can contain the modified sound profile information andinformation identifying the user, as well as any information entered bythe user about himself/herself and information about the audioreproduction that resulted in the modifications. The database identifiesthe user of the modified sound profile (1210). Then the databaseanalyzes the information accompanying the sound profile (1215). Thedatabase stores the modified sound profile for later use in response torequests from the user (1220). In addition, the database analyzes theuser's modifications to the sound profile compared to theparsed/analyzed data (1225). If enough users modify a preexisting soundprofile in a certain way, the preexisting default profile may be updatedaccordingly (1230). By way of example, if enough users from a certaingeography consistently increase the level of bass in a preexisting soundprofile for a certain genre of music, the preexisting sound profile forthat geography may be updated to reflect an increased level of bass. Inthis way, the database can be responsive to trends among users, andenhance the sound profile performance over time. This is helpful, forexample, if the database is being used to provide a streaming service,or other type of store where audio entertainment can be purchased.Similarly, if a user submits multiple sound profiles that have beenmodified in a similarly way (e.g. increasing the bass), the database canmodify the default profiles when the same user makes requests for newsound profiles. After a first user has submitted a handful of modifiedprofiles, the database can match the first user's changes to a seconduser in the database with more modified profiles and then use the seconduser's modified profiles when responding to future requests from thefirst user. The steps described in FIG. 12 need not be performed in theorder recited and two or more steps can be performed in parallel orcombined.

FIG. 13 shows a block diagram of a computer system capable of performingthe steps depicted in FIGS. 11 and 12. A subset of components incomputer system 1300 could also be used, and the components could befound in a PC, server, or cloud-based system. Bus 1365 can include oneor more physical connections and can permit unidirectional oromnidirectional communication between two or more of the components inthe computer system 1300. Alternatively, components connected to bus1365 can be connected to computer system 1300 through wirelesstechnologies such as Bluetooth, Wifi, or cellular technology. Thecomputer system 1300 can include a microphone 1345 for receiving soundand converting it to a digital audio signal. The microphone 1345 can becoupled to bus 1365, which can transfer the audio signal to one or moreother components. Computer system 1300 can include a headphone jack 1360for transmitting audio and data information to headphones and otheraudio devices.

An input 1340 including one or more input devices also can be configuredto receive instructions and information. For example, in someimplementations input 1340 can include a number of buttons. In someother implementations input 1340 can include one or more of a mouse, akeyboard, a touch pad, a touch screen, a joystick, a cable interface,voice recognition, and any other such input devices known in the art.Further, audio and image signals also can be received by the computersystem 1300 through the input 1340.

Further, computer system 1300 can include network interface 1320.Network interface 1320 can be wired or wireless. A wireless networkinterface 1320 can include one or more radios for making one or moresimultaneous communication connections (e.g., wireless, Bluetooth, lowpower Bluetooth, cellular systems, PCS systems, or satellitecommunications). A wired network interface 1320 can be implemented usingan Ethernet adapter or other wired infrastructure.

Computer system 1300 includes a processor 1310. Processor 1310 can usememory 1315 to aid in the processing of various signals, e.g., bystoring intermediate results. Memory 1315 can be volatile ornon-volatile memory. Either or both of original and processed signalscan be stored in memory 1315 for processing or stored in storage 1330for persistent storage. Further, storage 1330 can be integrated orremovable storage such as Secure Digital, Secure Digital High Capacity,Memory Stick, USB memory, compact flash, xD Picture Card, or a harddrive.

Image signals accessible in computer system 1300 can be presented on adisplay device 1335, which can be an LCD display, printer, projector,plasma display, or other display device. Display 1335 also can displayone or more user interfaces such as an input interface. The audiosignals available in computer system 1300 also can be presented throughoutput 1350. Output device 1350 can be a speaker. Headphone jack 1360can also be used to communicate digital or analog information, includingaudio and sound profiles.

In addition to being capable of performing virtually all of the samekinds of analysis, processing, parsing, editing, and playback tasks ascomputer system 400 described above, computer system 1300 is alsocapable of maintaining a database of users, either in storage 1330 oracross additional networked storage devices. This type of database canbe useful, for example, to operate a streaming service, or other type ofstore where audio entertainment can be purchased. Within the userdatabase, each user is assigned some sort of unique identifier. Whetherprovided to computer system 1300 using input 1340 or by transmissionsover network interface 1320, various data regarding each user can beassociated with that user's identifier in the database, includingdemographic information, geographic information, and informationregarding reproduction devices and consumption modalities. Processor1310 is capable of analyzing such data associated with a given user andextrapolate from it the user's likely preferences when it comes to audioreproduction. For example, given a particular user's location and age,processor 1310 may be able to extrapolate that that user prefers a morebass-intensive experience. As another example, processor 1310 couldrecognize from device information that a particular reproduction deviceis meant for a transportation modality, and may therefore require basssupplementation, time delays, or other 3D audio effects. These userreproduction preferences can be stored in the database for laterretrieval and use.

In addition to the user database, computer system 1300 is capable ofmaintaining a collection of sound profiles, either in storage 1330 oracross additional networked storage devices. Some sound profiles may begeneric, in the sense that they are not tied to particular, individualusers, but may rather be associated with artists, albums, genres, games,movies, geographical regions, demographic groups, consumptionmodalities, device types, or specific devices. Other sound profiles maybe associated with particular users, in that the users may have createdor modified a sound profile and submitted it to computer system 1300 inaccordance with the process described in FIG. 12. Such user-specificsound profiles not only contain the user's reproduction preferences but,by containing audio information and device information, they allowcomputer system 1300 to organize, maintain, analyze, and modify thesound profiles associated with a given user. For example, if a usermodifies a certain sound profile while listening to a particular song inthe user's car and submits that modified profile to computer system1300, processor 1310 may recognize the changes user has made and decidewhich of those changes are attributable to the transportation modalityversus which are more generally applicable. The user's other preexistingsound profiles can then be modified in ways particular to theirmodalities if different. Given a sufficient user population, then,trends in changing preferences will become apparent and processor 1310can track such trends and use them to modify sound profiles moregenerally. For example, if a particular demographic group's reproductionpreferences are changing according to a particular trend as they age,computer system 1300 can be sensitive to that trend and modify all theprofiles associated with users in that demographic group accordingly.

In accordance with the process described in FIG. 11, users may requestsound profiles from the collection maintained by computer system 1300,and when such requests are received over network interface 1320,processor 1310 is capable of performing the analysis and extrapolationnecessary to determine the proper profile to return to the user inresponse to the request. If the user has changed consumption modalitiessince submitting a sound profile, for example, that change may beapparent in the device information associated with the user's request,and processor 1310 can either select a particular preexisting soundprofile that suits that consumption modality, or adjust a preexistingsound profile to better suit that new modality. Similar examples arepossible with users who use multiple reproduction devices, changegenres, and so forth.

Given that computer system 1300 will be required to make selectionsamong sound profiles in a multivariable system (e.g., artist, genre,consumption modality, demographic information, reproduction device),weighting tables may need to programmed into storage 1330 to allowprocessor 1310 to balance such factors. Again, such weighting tables canbe modified over time if computer system 1300 detects that certainvariables are predominating over others.

In addition to the user database and collection of sound profiles,computer system 1300 is also capable of maintaining libraries of audiocontent in its own storage 1330 and/or accessing other, networkedlibraries of audio content. In this way, computer system 1300 can beused not just to provide sound profiles in response to user requests,but also to provide the audio content itself that will be reproducedusing those sound profiles as part of a streaming service, or other typeof store where audio entertainment can be purchased. For example, inresponse to a user request to listen to a particular song in the user'scar, computer system 1300 could select the appropriate sound profile,transmit it over network interface 1320 to the reproduction device inthe car and then stream the requested song to the car for reproductionusing the sound profile. Alternatively, the entire audio filerepresenting the song could be sent for reproduction.

FIG. 14 shows a diagram of how computer system 1300 can service multipleusers from its user database. Computer system 1300 communicates over theInternet 140 using network connections 150 with each of the usersdenoted at 1410, 1420, and 1430. User 1410 uses three reproductiondevices, head end 111, likely in a transportation modality, stereo 115,likely in an indoor modality, and portable media player 110, whosemodality may change depending on its location. Accordingly, when user1410 contacts computer system 1300 to make a sound profile request, thedevice information associated with that request may identify which ofthese reproduction devices is being used, where, and how to help informcomputer system 1300's selection of a sound profile. User 1420 only hasone reproduction device, headphones 200, and user 1430 has threedevices, television 113, media player 114, and videogame system 116, butotherwise the process is identical.

Playback can be further enhanced by a deeper analysis of a user's musiclibrary. For example,

In addition to more traditional audio selection metrics such as artist,genre, or the use of sonographic algorithms, intensity can be used as acriteria by which to select audio content. In this context, intensityrefers to the blending of the low-frequency sound wave, amplitude, andwavelength. Using beats-per-minute and sound wave frequency, each filein a library of audio files can be assigned an intensity score, e.g.,from 1 to 4, with Level 1 being the lowest intensity level and Level 4being the highest. When all or a subset of these audio files are loadedonto a reproduction device, that device can detect the files (1505) anddetermine their intensity, sorting them based on their intensity levelin the process (1510). The user then need only input his or her desiredintensity level and the reproduction device can create a customizedplaylist of files based on the user's intensity selection (1520). Forexample, if the user has just returned home from a hard day of work, theuser may desire low-intensity files and select Level 1. Alternatively,the user may be preparing to exercise, in which case the user may selectLevel 4. If the user desires, the intensity selection can beaccomplished by the device itself, e.g., by recognizing the geographiclocation and making an extrapolation of the desired intensity at thatlocation. By way of example, if the user is at the gym, the device canrecognize that location and automatically extrapolate that Level 4 willbe desired. The user can provide feedback while listening to theintensity-selected playlist and the system can use such feedback toadjust the user's intensity level selection and the resulting playlist(1530). Finally, the user's intensity settings, as well as the iterativefeedback and resulting playlists can be returned to the computer systemfor further analysis (1540). By analyzing user's responses to theselected playlists, better intensity scores can be assigned to eachfile, better correlations between each of the variables (BPM, soundwavefrequency) and intensity can be developed, and better predictionpatterns of which files users will enjoy at a given intensity level canbe constructed.

The steps described in FIG. 15 need not be performed in the orderrecited and two or more steps can be performed in parallel or combined.The steps of FIG. 15 can be accomplished by a user's reproductiondevice, such as those with the capabilities depicted in FIGS. 3 and 4.Alternatively, the steps in FIG. 15 could be performed in the cloud oron a server on the Internet by a device with the capabilities of thosedepicted in FIG. 13 as part of a streaming service or other type ofstore where audio entertainment can be purchased. The intensity analysiscould be done for each song and stored with corresponding metadata foreach song. The information could be provided to a user when it requestsone or more sound profiles to save power on the device and create a moreconsistent intensity analysis. In another aspect, an intensity scorecalculated by a device could be uploaded with a modified sound profileand the sound profile database could store that intensity score andprovide it to other users requesting sound profiles for the same song.

FIGS. 16A-B show an exemplary user interface by which the user canperform intensity-based content selection on a reproduction device suchas mobile device 110. In FIG. 16A, the various intensity levels arerepresented by color gradations 1610. By moving slider 1620 up or down,the user can select an intensity level based on the colorrepresentations. Metadata such as artist and song titles can be layeredon top of visual elements 1610 to provide specific examples of songsthat match the selected intensity score. In FIG. 16B, hapticinterpretations have been added as concentric circles 1630 and 1640. Byvarying the spacing, line weight, and/or oscillation frequency of thesecircles, a visual throbbing effect can be depicted to represent changesin the haptic response at the different intensity levels so the user canselect the appropriate, desired level. As one skilled in the art willappreciate, the controls of the interface presented in FIGS. 16A and 16Bcould be accomplished with alternative tools. FIGS. 3 and 4 show systemscapable of providing the user interface depicted in FIGS. 16A-B.

FIG. 17 shows exemplary steps performed by a device to determineparameters for a sound profile based on a user's sensitivity acrossvarious frequency ranges.

The steps shown in FIG. 17 can be performed by the device shown in FIG.4. The devices contain a computer system 400 that can be used toimplement the techniques described herein for playing audio files. Thecomputer system 400 can include the network interface 420 used totransmit and receive data over a network. Alternatively, the networkinterface 420 can be used locally to transmit and receive data withoutgoing through a network.

For example, the network interface 420 is capable of transmittinginformation about the computer system 400 for reproducing enhanced mediacontent, audio metadata related to the audio data in the media content,and a sound profile over a network, where the enhanced media content,audio metadata, and the sound profile are stored in the memory 415.

For example, the network interface 420 can receive a sound profile froma network server, not shown, or from the memory 415 where the soundprofile is stored. The sound profile can contain initial preselectedparameters for modifying the audio data, where one or more preselectedparameters in the sound profile can be matched to one or more pieces ofinformation in the audio metadata stored in the memory 415 related tothe audio data in the media or the information about the device forreproducing enhanced media content. The sound profile can be used by theprocessor 410 to generate customized audio playback. Furthermore, theone or more preselected parameters for modifying the audio data arereceived by the network interface 420 without previously transmitting auser's parameters for modifying the audio data from the device.

As another example, the computer system 400 can include the processor410 which can request sound profiles from a network such as the Internetthrough network interface 420 or from storage 430 for the specific song,game or video based on the artist, genre, or specific song or video.Processor 410 can provide information through the network interface 420to allow selection of a sound profile based on device information suchas geography, user ID, user demographics, device ID, consumptionmodality, the type of reproduction device (e.g., mobile device, headunit, or Bluetooth speakers), reproduction device, or speakerarrangement (e.g., headphones plugged or multi-channel surround sound).Processor 410 can then use input received from input 440 to modify asound profile according to a user's preferences. Processor 410 can thentransmit the sound profile to a headphone connected through networkinterface 420 or headphone jack 460 and/or store a new sound profile instorage 430. Processor 410 can run applications on computer system 400like Alpine's Tune-It mobile application, which can adjust soundprofiles.

A common object used by components of the computer system 400 is thesound profile. For example, a sound profile is received and transmittedby the network interface 420, and saved in the memory 415, or evenstored in a network server, not shown. The processor 410 can furtherdetermine how to play an audio file based on the sound profile.

As discussed above, sound profile can include data related todemographic information, ethnicity information, age information, socialmedia information, and previous usage information of the user. Suchinformation can be gathered from internet over time, from variouswebsites such as social media websites, or other public sources. Suchinformation can be useful to determine and predict what kind of soundprofile the user may want to start without the user providing anyinformation on the sound profile. Alternatively, a user can provide suchinformation as well. A user of different demography, ethnicity, age canhave different preference in how an audio file should be played. On theother hand, users of similar demography, ethnicity, age can have similarpreference in how an audio file should be played. Recent previous usagecan be a good indicator of what the user may currently enjoy listening.On the other hand, the sound profile of a user can change over time whenone ages.

A sound profile can also be device specific to include information forvarious devices used by a same user. For example, a sound profile can bedifferent for a mobile device 110, head unit 111, stereo 115, as shownin FIG. 4. A sound profile can have location information as well sincedifferent sound preference can be needed for playing the audio file indifferent locations. For example, a lower volume is preferred when theaudio file is played inside a bedroom, compared to the volume neededwhen the audio file is played in a noisy bus stop. When there is ahaptic device connected to the device playing the audio file, the soundprofile can include the haptic movement of the audio file as well.

A sound profile can be specific for each audio file, or can be a generalprofile about a group of audio files. For example, the sound profile caninclude the intensity score of an audio file. The sound profile is basedon individual user. It can be specific to left ear or right ear when onehas a different hearing profile on the left ear or right ear. It can beapplied to both left ear and right ear as well.

More information can be included in a sound profile which will bedescribed below, which can include an inaudible frequency range and howthe lost frequency range is compensated, such as by haptic movement.

FIG. 17 shows exemplary steps performed by a device to determineparameters for a sound profile based on a user's sensitivity acrossvarious frequency ranges. The exemplary steps can be performed by thedevice shown in FIG. 4. The exemplary steps can be used to add to asound profile an inaudible frequency range that is inaudible at a highfrequency, and add to the sound profile another inaudible frequencyrange at a low frequency, with respect to an ear (right ear or left ear)of a user. Inaudible frequency ranges can be added for both ears. Theinaudible frequency ranges can be added to a user's sound file orrelated to an audio file. The exemplary steps can further add to thesound profile a lowest audible volume for one or more audible frequencyrange.

At 1705, the network interface 420 can receive a sound profile,including data about a location, a device, left ear/right ear, socialprofile, previous usage data, haptic, demographic, ethnicity, andfrequency ranges. More details of the sound profile have been describedabove. The sound profile can be received over a network, or receivedlocally from memory 415.

At 1710, the circuitry in processor 410 can be configured to divide anaudio frequency region into a number of frequency ranges. Such anexample of dividing an audio frequency region into a number of frequencyranges is shown in FIGS. 18A-C, where a frequency region of [20 hz-16khz] is divided into 7 frequency ranges. Additional divisions are alsopossible. The frequency region can be divided into bark bands orcritical bands. More details will be described below.

At 1715, the processor 410 can be configured to select a frequency rangeamong the number of frequency ranges to perform the following work. Theselection of a frequency range can be based on a user's input.Alternatively, the selection of a frequency range can be done in apredetermined way that is programmed or decided by the device. Forexample, the selection of the frequency range can start from the lowestfrequency range within the frequency region. Alternatively, theselection can start from the highest frequency range within thefrequency region. The frequency range can also be selected based oninformation in the sound profile, such as demographic information.

At 1720, the processor 410 can be configured to test whether theselected frequency range is audible or not. More details will bedescribed in FIGS. 19 and 21A-C.

At 1725, if the selected range is audible, the processor 410 can beconfigured to set the lowest audible volume for the frequency range.

At 1730, the processor 410 can be configured to compensate the firstinaudible frequency range under a first set of compensation parameters.Compensating the first inaudible frequency range for the audio file canbe done in different ways. For example, compensating the first inaudiblefrequency range for the audio file is performed by generating hapticmovement corresponding to the first inaudible frequency range for theaudio file. Therefore even when the user cannot hear the sound in thefirst inaudible frequency range, the user can feel the tactile sensationgenerated by movement of the haptic device connected to the deviceplaying the sound.

As another example, compensating the first inaudible frequency range isperformed by compressing the audio data from the inaudible frequencyranges into neighboring audio frequency ranges. The compressiontechniques work by splitting the full-range frequency region into anumber of frequency range, and if a frequency range is inaudible, thenthe compression technique can compensate the missing data in theinaudible range by altering the intensity in the neighboring, audiblefrequency range. The frequency ranges can be set based on critical bandsor bark bands. Other forms of compression can also include wide dynamicrange compression, bass increase at low levels, treble increase at lowlevels,

In another example, an inaudible frequency range can also be subdividedinto smaller ranges, with each smaller range then tested as a separatefrequency range according to the process in FIG. 17, so that only thenarrowest inaudible frequency range is identified and compensated

Other methods of compensating the first inaudible frequency range, suchas selective gain, can be used as well. Selective gain varies theamplification for a given frequency range based on the input signals.For example, if the input signal for a given frequency range is alreadyloud, less amplification is given; if the input signal is soft, moreamplification is given. The frequency ranges can be set based oncritical bands or bark bands. The frequency ranges can also be furthersubdivided.

As another compensation technique, a sound signal can be described bythe frequency ranges and its dynamic range, which is the volume oramplitude in a frequency range. The compensation method for the firstinaudible frequency range can move the audio data in the inaudiblefrequency range up to the next frequency range that is audible, whileamplifying the amplitude of the data as well. The frequency ranges canbe set based on critical bands or bark bands, and the amplifying can beperformed to compensate for the quieting that occurs due to spreadingbetween critical bands or bark bands.

At 1735, the processor 410 can be configured to test whether allfrequency ranges have been tested. If not, the frequency range can beincremented to the next frequency (1740), either up or down, and thetesting process starts again. The next frequency range to be selected byother ways too.

At 1745, the processor 410 can be configured to update the sound profileto include the inaudible frequency ranges for the user or audio file,which can help to produce sound effect better fits into the user's ear.The sound profile can also include how the inaudible frequency rangesare compensated for the user based on the set of compensation parametersdetermined by the processor 410 in step 1730. The processor 410 can beconfigured to update at step 1745 the sound profile with the frequencyrange and the lowest audible volume for each frequency range that isaudible.

At 1750, the processor 410 can be configured to modify the audio file inthe media content according to the updated sound profile, access thestored media content including the audio data from the memory componentand the information about the device for reproducing enhanced mediacontent, and playback the audio file based on the updated sound profile,the information about the device, and the media content.

At 1760, once all the frequency ranges have been set up, the devicedisplays the frequency ranges that are audible. More details can beshown in FIGS. 21A-C.

The sequence of actions shown in chart 1700 is only for example and notlimiting. Some steps can be omitted while additional steps can be added,which will be within the scope of the disclosure. For example, someadditional steps on haptic movement related to the sound profile can beadded, which will be described in FIG. 20.

FIGS. 18A-C show an exemplary audible frequency region divided intodifferent frequency ranges, some of which may be inaudible to individualusers. Those are examples of the step 1710 shown in FIG. 17. In anotherexample, the frequency ranges can be divided into critical bands or barkbands, or into groupings of critical bands or bark bands.

FIG. 18A shows a graph illustrating the audible frequency region and thecorresponding audible sound pressure level (db) sensitivity for anaverage human user. Individual user can have different audible frequencyregion and sound pressure level sensitivities. As shown in FIG. 18A,generally a human user can hear a loud sound around 20 hz, as shown inpoint 1810 and a quieter sound around 15 khz as shown in point 1805. Foreach frequency point, the audible sound pressure level can be different.For example, at the low frequency end, around 20 hz shown in 1810, asound is audible at a sound pressure level of around 60 db, while asound is audible at a sound pressure level of 10 db around 4 kHz shownin 1815. Different users will have different levels of perception fordifferent frequencies. Therefore it is important and useful to customizethe audible frequency region and sound pressure level for eachindividual user and save such information into the sound profile for theuser. FIG. 18A is only for example purpose and not limiting.

FIG. 18B shows the generally audible frequency region divided intodifferent frequency ranges, as discussed with respect to step 1710 inFIG. 17. As shown in FIG. 18B, the possible audible region 20 hz to 16khz is divided into 7 different frequency ranges shown in column 2,while a sample frequency point in the range is shown in column 3. Thedivision of the 7 ranges is not equally spaced. For example, the lowestfrequency range is at 20 hz to 100 hz shown at Range No. 1 and thesecond column, while the highest frequency range is at 10 khz to 16 khzshown at Range No. 7 and the second column. Such frequency rangedivision is based on experience and experiments. Different frequencyrange division is possible, such as Bark bands, critical bands, orgroupings of those bands. Dividing the audible frequency region intoranges is not irreversible. In some examples, where a user's sensitivityto a frequency range is unclear, the range can be subdivided to betterassess the users hearing in the subdivided ranges.

FIG. 18C shows how the audible frequency region can change with the agefor a user. Therefore the age of a user can be included in a soundprofile as an indication of the possible audible frequency ranges. Theage of a user can also suggest default settings for audibility settingsin sound profiles. As shown in region 1825 in FIG. 18C, for the high endfrequency, one can possibly hear really high end frequencies around 20khz when one is younger than 18 years old. The high end audiblefrequency range is reduced to about 17 khz when one reaches 24 years oldas shown at point 1835. The high end audible frequency range slowlydecreases with age, to around 15 khz when one reaches 40 years old asshown at point 1840. Afterwards the high end audible frequency range canaccelerate the loss to 12 khz when one reaches 50 years old as shown atpoint 1845. The speed of change of audible frequency region can varywith age. For example, the region 1830 between age 18-24 and between theage 40-50 are the region where the hearing loss can be acceleratedcompared to other age periods. The information illustrated in FIG. 18Cis for general demonstration of a relationship between the high endaudible frequency range with the age of a user. Individuals may have adifferent profile of audible frequency range changing with age. Theinformation illustrated in FIG. 18C shows the usefulness of detectingthe inaudible frequency range for a user and compensate the lost audiblefrequency range in order to get a better audio experience, since a soundprofile for a user can change over time when one ages.

FIG. 19 shows exemplary steps performed by a device to detect aninaudible frequency range and to generate a lowest volume in an audiblefrequency range for individual users. The exemplary steps can also beused to update a sound profile.

At 1715, the device displays a frequency range not set up yet and selectthe frequency range to be set up. The frequency range is a part of anaudible frequency region for a general user. To set up a frequency rangeis to detect whether the frequency range is audible or not. If thefrequency range is audible, the device detects the lowest audible volumefor the frequency range. The device can divide an audio frequency regioninto a number of frequency ranges, as shown in FIG. 18B. The device canfurther display a frequency range for a user to select to set up. Thefrequency range displayed can be based on a user's input. Alternatively,the frequency range displayed can be selected in a predetermined waythat is programmed or decided by the device. For example, the frequencyrange displayed can start from the lowest frequency range within thefrequency region. Alternatively, the frequency range displayed can startfrom the highest frequency range within the frequency region. Examplesof such displayed frequency range is shown in FIG. 21A-21C.

A user can select the displayed frequency range to set up the lowestaudible volume for the audio file. The selection of the displayedfrequency range can be done by pressing the display to make theselection. Other methods such as clicking, or tap the display can beused as well. One such a selection example is shown in FIGS. 21A-C.

The steps 1915-1935 are detailed steps of the step 1720 of FIG. 17,which tests whether the selected frequency range is audible or not,while setting a lowest audible volume for the frequency range if it isaudible, and further updating a sound profile.

At 1915, the user can increase the sound volume at the selectedfrequency range. When the frequency range is initially displayed, thedevice is set to play an audio file at a really low volume so that auser is unlikely to hear the sound in the frequency range at the initialsound volume. When the user increases the volume of the audio file beingplayed, the user may be able to hear the sound. The increase of theaudio volume can be done by pressing a button or a key continuously. Itcan also be set digitally if a digital display of the volume isavailable. Furthermore, the audio file being used in the testing of thelowest audible volume can be some pink noise, or some classic musicsound, or an audio file the user chooses.

At 1920, when there is a haptic device connected to the device playingthe audio file, the device playing the audio file can send signals tothe haptic device indicating the currently chosen volume, while thehaptic device will generate haptic movement corresponding to thecurrently chosen sound volume. When the user continuously increases thevolume being played, the haptic movement changes at a same time when thevolume being played changes. In this way, the user can feel the audiofrequency range being played in addition to listen if it is audible.Some device playing the audio file may not be connected to a hapticdevice, in which case no haptic movement will be generated.

At 1925, the user will monitor whether a sound is heard or not. The userwill continue to increase the sound volume until it becomes audible by atesting ear, in which case the selected frequency range is audible, andthe user has found the lowest audible volume for the frequency range.Alternatively, the user will continue to increase the sound volume untilit exceeds the maximum allowed volume for the frequency range.

At 1935, if the user keeps increasing the sound volume at the frequencyrange, and still cannot hear anything, the device will monitor whetherthe volume has exceeded the maximum allowed volume for the audio file inthe frequency range. If the maximum allowed volume has not been reachedyet, the user can keep increasing the volume and the device loops backto step 1915.

At 1725, the process is as described above for step 1725 in FIG. 17.

Alternatively, at 1730, the process is as described above for step 1730in FIG. 17. when the audio volume being increased exceeds apredetermined volume, the device can determine the selected frequencyrange is inaudible to the user. The predetermined volume can be set bythe device in advance. The device can display an indication that thefrequency range is inaudible. The indication can be a sound, or a worddescription of such.

The device goes through the frequency ranges found to be inaudible,starting from the lowest frequency range until it reaches a firstauditable frequency range. The frequency range right before the firstaudible frequency range is the first inaudible frequency range that isinaudible at a low frequency end.

Similarly, the device goes through the frequency ranges found to beinaudible, starting from the highest frequency range until it reaches afirst auditable frequency range. The frequency range right before thefirst audible frequency range is the first inaudible frequency rangethat is inaudible at a high frequency end.

FIG. 20 shows exemplary steps performed by a device to detect hapticsensitivity across various frequency ranges. In some example, a hapticdevice can be placed on a human body, as shown in FIGS. 22A-B. When thehaptic device is placed at different parts of a human body, theintensity needed for the haptic movement to be detectable by the humanbody part can be different. The processor 410 can be similarlyconfigured to generate the lowest intensity for a haptic device to havea movement detectable on a human body in the frequency range forindividual users. The detection of the haptic device movement can bedone by a sensor within or connected to the haptic device, or by human'sown sense and feel. These steps in FIG. 20 can be easily combined withthe steps shown in FIG. 17.

At 2005, the device displays a frequency range not set up yet andselects the frequency range to be set up for haptic movement related toa haptic device placed on a contact point of a human body. The frequencyrange can be a part of an audible frequency region for a general user.In this step, to set up a frequency range is to decide whether thehaptic device movement can be detected or not, by a sensor or by thehuman. The device detects the lowest intensity of the haptic device tohave a movement detectable on a human body for the frequency range. Auser can select a frequency range to set up. The selection of thefrequency range can be done manually or by the device itself in apredetermined manner.

At 2015, the device can send signals to a connected haptic device,indicating the currently chosen intensity, while the haptic device willgenerate haptic movement corresponding to the currently chosenintensity.

At 2020, the user can increase the haptic intensity at the selectedfrequency range. When the frequency range is initially displayed, thehaptic device is set to play at a really low intensity so that a user isunlikely to feel the haptic movement in the frequency range, while thesound can be played at a low volume. With the increase of the overallintensity, the intensity of the haptic device for a given frequency canbe increased as well.

At 2025, the user will monitor whether a haptic movement is detectableor not. The detection can be done by the sense of human body part wherethe haptic device is attached. Alternative, the detection of the hapticmovement is done by a sensor to collect the feedback from the human bodypart. The user will continue to increase the volume or the hapticmovement intensity until it becomes detectable. Alternatively, the userwill continue to increase the speed until it exceeds the maximum allowedintensity for the frequency range.

At 2035, if the user keeps increasing the intensity of the haptic deviceat the frequency range, and still cannot detect the haptic movement, thedevice will monitor whether the intensity has exceeded the maximumallowed intensity in the frequency range. If the maximum allowedintensity has not been reached yet, the user can keep increasing theintensity and the device loops back to step 2015.

At 2040, after the user has found the lowest intensity for a hapticdevice to have a movement detectable on a human body for the frequencyrange, the device can record the lowest intensity for the selectedfrequency range, and update the sound profile accordingly.

Alternatively, at 2045, when the intensity being increased exceeds apredetermined intensity, the device can determine the selected frequencyrange is not detectable to the user at the body part the haptic deviceis placed. The predetermined intensity can be set by the device inadvance. The device can decide how to compensate for the frequency rangewithin which no haptic movement can be detected.

FIGS. 21A-C show a user interface for detecting a user's sensitivity tosounds or haptic movement across various frequency ranges. Similarly,the same actions can be used to generate a lowest intensity for a hapticdevice to have a movement detectable on a human body in a frequencyrange for individual users. The sequence of actions can be played in adevice 400 shown in FIG. 4. The user interface shown in the sequence ofactions indicates the frequency range, facilitates the change of volumeof the sound file or the intensity of a haptic device, and providesfeedback when the lowest volume or intensity is detected.

As shown in FIG. 21A, the display contains one circle 2105 which showsone frequency range not set up yet. The display can be for a device asshown in FIG. 4, and controlled by the processor 410. In fact, there aremultiple frequency range not set up yet, but the device can display onesuch frequency range. The display can also contain a word description2130 to remind the user how to set up a frequency range. The user canfollow the word description to set up the frequency range, which can bedone similarly as shown previously in step 1720. For example, as shownin FIG. 21A, the word description 2130 states that “hold down on thepulsing sound pad until you start to hear the note,” which instructs theuser to hold down the pulsing sound pad to increase the volume, andreleases the pulsing sound pad until a note is heard.

As shown in FIG. 21B, if the frequency range represented by the circle2105 is audible, and the device detects the lowest audible sound for thefrequency range, the device will display the same circle 2105 in adifferent color or shape to indicate it represents an audible frequencyrange. In addition, after the first circle 2105 is set up, the displaycan contain a second circle 2110 in addition to the first circle 2105.The user can set up the frequency range represented by the circle 2110similarly as before. For example, the user can hold down the pulsingsound pad to increase the volume, and releases the pulsing sound paduntil a note is heard.

As shown in FIG. 21C, when all the frequency ranges have been set up,there are three more frequency ranges represented by the circle 2115,circle 2120, and circle 2125 displayed to show the audible frequencyranges. Overall, there can be a total of 7 frequency ranges as shown inFIG. 18B, where two frequency ranges are inaudible. Such inaudiblefrequency ranges are not shown in the display. In another example, asdiscussed above, there can be more frequency ranges.

It is only for example to represent a frequency range as circles such asthose used in FIGS. 21A-C. Other shapes can be used as well. Inaddition, other visual effects such as animated circles or shapes can beused to invite the user to set up a frequency range. The device maydisplay multiple frequency ranges that are not set up, and let the userto choose which frequency range to set up first.

FIGS. 22A-B shows an exemplary haptic device and its components. Suchhaptic devices can be connected to the device playing audio files, andgenerate haptic movement based on the sound generated by the deviceplaying audio files, as shown in step 2015 in FIG. 20. Alternatively,they can be connected to devices relaying other signals. Haptic movementcan be used to compensate the inaudible frequency ranges for a user, andcan be used to enhance the audio experience in an audible frequencyrange for a user. Alternatively, a haptic device can be used alonecontrolled by the audio data without reproducing acoustically the audiofiles containing the audio data.

As shown in FIG. 22A, a haptic device 2200 can contain a physicalsupport 2205, an attachment component 2210, a communication interface2215 to receive audio signals from a device playing audio files, amechanical movement component 2230, a processing unit 2220 to controlthe movement by sound, and a sensor 2225 to collect feedback of the partwhere the haptic device is attached. The components shown in FIG. 22Aare for example only, and are not limiting. Some components can beomitted while other components can be added to a haptic device.

The physical support 2205 is the physical component where other partssuch as the mechanical movement component 2230 and the processing unit2220 are placed and fixed. For example, the headphone 200 can be such aphysical support 2205. However, the physical support 2205 is not limitedto a headphone. Instead, it can be a loop where other components areplaced, where the loop is made of fabric or metal. In general, thephysical support 2205 can have a surface big enough to have othercomponents arranged in a position that is relatively stable. The size ofthe surface of the physical support 2205 can be about a same size as thesum of other component surface sizes. The physical support 2205 can bein various shapes such as a loop, a flat surface, or a cylinder shape.

The attachment component 2210 is used to attach the haptic device 2200to a human body part or other part. It connects and fixes the hapticdevice into affixed position on a human body. It is attached to thephysical support 2205. The attachment component 2210 and the physicalsupport 2205 can be made by metal, leather, and other materials. Theattachment component 2210 makes the connection between the haptic deviceand a human body. For example, it can be a hat, headband, watch,bracelet, or a shoe.

The mechanical movement component 2230 is used to generate themechanical movement for the haptic device in contact with a human bodyor other surface. The mechanical movement component 2230 can be in touchwith a human body and moves systemically under the control of theprocessing unit 2220. The mechanical movement can be in one uniformeddirection, or in more than one directions. It can be with differentforces at different times.

The processing unit 2220 is used to control the movement of themechanical movement component 2230. The processing unit 2220 can controlthe mechanical movement by the volume of a sound or frequency of thesignal or audio files been played. For example, the processing unit 2220can increase the intensity of the mechanical movement of the movementcomponent 2230 when the volume of a sound is high, and reduce theintensity when the volume is low. Alternatively, the processing unit2220 can also process the received audio data without the audio filebeen played.

Finally, the sensor 2225 is used to collect feedback of the contactpoint of the human body. When the mechanical movement component 2230makes movement based on a signal or the audio data, the generatedmovement can create impact on the human body. For example, the humanbody can feel the temperature increases when the intensity of themechanical movement increases. Alternatively, the blood may flow fasterat the contact point when the intensity of the mechanical movementincreases. The senor can detect such human body reactions or feedbacksimpacted by the mechanical movement of the haptic device 2200. Suchfeedback can be sent back to the audio device playing the signals audiofile to control the haptic device 2200. The sensor 2225 can send suchfeedback data in real time so that the device playing the signals oraudio file can adjust the sound profile based on the collected feedback.

As shown in FIG. 22B, a haptic device can be placed at various positionsof a human body. For example, the haptic device can be placed at the topof the head 2240, near an ear 2245, on the arm 2250, around the leg2255, or around the wrest 2260. For example, the haptic device can beplaced on a hat or a headband when it is placed at the top of the head2240. The haptic device can be placed on a watch or bracelet wore on anarm 2250, attached to a shoe around the leg 2255, attached to a beltaround the wrest 2260, or attached to an earring next to an ear 2245.Alternatively, in some other example, the haptic device can be placeddirectly on the skin, or even within a human body such as a medicaldevice.

Even through a haptic device is placed on a human body in examples givenin this disclosure, a haptic device can be placed on other non-humanbody such as on an animal body for medical or industry applications.

When a haptic device is placed at a different human body part, thesensitivity and the preferred mechanical intensity for the haptic devicecan be different. Therefore a user can adjust the haptic intensity ofthe haptic device depending on the location of the device on the humanbody, where the haptic intensity can be controlled by an audio volume ordirectly by some audio data. The details of the adjustment and controlof the haptic intensity of the haptic device for each location is shownin FIG. 20.

FIG. 23 shows exemplary steps performed by a device to predict and use asound profile. At 2305, the device can collect information of the user.Previously in the step 1805 shown in the method 1800, the devicereceives a sound profile, without a user input any parameter formodifying the audio data. At 2305, the method 2300 can collectinformation from the user about user's preference on sound profiles. Theinformation provided by the user may be partial and incomplete, whichcan serve as a base for the device to predict a more complete soundprofile matches the user information.

The user provided information can include a first inaudible frequencyrange at a low frequency end and a second inaudible frequency range at ahigh frequency end, and methods for compensating the inaudible frequencyranges. The sound profile can further include audio frequency ranges andthe lowest audible volume in each audible frequency range.

At 2310, the device predicts a sound profile, based on the collecteduser information, an audio file, a location, a device, left ear/rightear, social profile, previous usage data, haptic, demographic,ethnicity, inaudible frequency range, and audible frequency ranges. Thepredicted sound profile can be more complete than the informationprovided by the user at step 2305.

At 2315, the device can play the audio file based on the predicted soundprofile. When the device plays the audio file based on the predictedsound profile, the device can compensate a first inaudible frequencyrange at a low frequency end and a second inaudible frequency range at ahigh frequency end, as outlined previously, such as in the descriptionfor the steps 1830 and 1840.

At 2320, a haptic device connected to the device playing the audio filecan generate haptic movement based on the audio file playback. Thehaptic device can be one described in FIG. 22A and placed on any partshown in FIG. 22B.

At 2325, the sensor within the haptic device can generate feedback basedon the impact on the part the haptic device is placed on. The hapticdevice can further send back the feedback to the device playing theaudio file, which can further revise the sound profile.

At 2330, the user can decide whether the sound profile is satisfactoryor not, based on the audio impact heard, and the haptic movement felt atthe body part. If the audio impact heard or the haptic movement felt isnot satisfactory, the device will go back to step 2305 to furthercollect user information and predict a different sound profile at step2310.

At 2335, the device can continue to play the audio file if the soundprofile is satisfactory, based on the audio impact heard, and the hapticmovement felt at the body part.

A number of examples of implementations have been disclosed herein.Other implementations are possible based on what is disclosed andillustrated.

What is claimed is:
 1. A device for reproducing enhanced media content,the media content including audio data, comprising: a memory componentcapable of storing media content, wherein the media content includesaudio data and audio metadata related to the audio data in the mediacontent; a network component capable of receiving a first sound profile,wherein the first sound profile is capable of containing initialpreselected parameters for modifying the audio data; wherein one or moreinitial preselected parameters in the first sound profile are matched toone or more pieces of information in the audio metadata related to theaudio data in the media or the information about the device forreproducing enhanced media content; wherein the one or more initialpreselected parameters for modifying the audio data are received withoutpreviously transmitting a user's parameters for modifying the audio datafrom the device; and a processor component configured to access thestored media content including the audio data from the memory componentand the information about the device for reproducing enhanced mediacontent, access the first sound profile received by the networkcomponent, determine a first inaudible frequency range that is inaudiblefor a user, compensate the first inaudible frequency range under a firstcompensation parameter, update the first sound profile with the firstcompensation parameter for the first inaudible frequency range to createa second sound profile, and modify the audio data in the media contentaccording to the second sound profile.
 2. The device of claim 1, whereinthe processor component is further configured to: determine a secondinaudible frequency range that is a different frequency range than thefirst inaudible frequency range; compensate the second inaudiblefrequency range under a second compensation parameter; update the secondsound profile with the second compensation parameter to create a thirdsound profile; and modify the audio data in the media content accordingto the third sound profile.
 3. The device of claim 1, wherein thenetwork component is further configured to transmit information aboutthe device for reproducing enhanced media content, audio metadatarelated to the audio data in the media content, and the second soundprofile over a network.
 4. The device of claim 1, wherein the processorcomponent is configured to compensate the first inaudible frequencyrange by being configured to generate a haptic movement corresponding tothe first inaudible frequency range.
 5. The device of claim 1, whereinthe processor component is configured to compensate the first inaudiblefrequency range by being configured to generate compressed audio datafrom the first inaudible frequency range into a next audible audiofrequency range for the audio data.
 6. The device of claim 1, whereinthe first sound profile comprises data related to haptic movement whichis specific to a left ear or a right ear, demographic information,ethnicity information, age information, location information, socialmedia information, intensity score of the audio data, previous usageinformation, or device information.
 7. The device of claim 1, wherein todetermine the first inaudible frequency range, the processor is furtherconfigured to: divide an audio frequency region into a number offrequency ranges; start from a selected frequency range among the numberof frequency ranges and test the selected frequency range is inaudibleor not; continue testing whether all of the frequency ranges areinaudible; update the sound profile with one or more parametersidentifying all of the inaudible frequency ranges.
 8. The device ofclaim 7, wherein, to test whether the selected frequency range isinaudible the processor is configured to: start from an audio volume atthe selected frequency range that is inaudible; increase the audiovolume at the selected frequency range until it becomes audible by atesting ear or exceeds a predetermined maximum volume; identify thefrequency range as inaudible if the predetermine maximum volume isreached.
 9. The device of claim 1, wherein the processor component isfurther configured to: divide an audio frequency region into a number offrequency ranges; select a frequency range that is audible; decrease orincrease an audio volume at the selected frequency range until itbecomes a lowest audible volume in the selected frequency range; recordthe lowest audible volume for the selected frequency range; and updatethe first sound profile with the lowest audible volume for the selectedfrequency range.
 10. The device of claim 1, wherein the processorcomponent is further configured to: play the audio data based on thefirst sound profile; generate haptic movement for a haptic deviceconnected to the device for reproducing enhanced media content; receivefeedback from a sensor connected to the haptic device; and revise thefirst sound profile based on the feedback received from the sensorconnected to the haptic device.
 11. The device of claim 10, wherein thehaptic device is wearable on a human body, and the haptic devicecomprises a mechanical movement component to generate a mechanicalmovement on the contact point, a processing unit to process receivedaudio haptic and to control the mechanical movement, and a sensor tocollect feedback from the contact point.
 12. A device for reproducingenhanced media content, the media content including audio data,comprising: a haptic device, a memory component capable of storing mediacontent, wherein the media content includes audio data and audiometadata related to the audio data in the media content; a networkcomponent capable of receiving a first sound profile, wherein the firstsound profile is capable of containing initial preselected parametersfor modifying the audio data; wherein one or more initial preselectedparameters in the first sound profile are matched to one or more piecesof information in the audio metadata related to the audio data in themedia or the information about the device for reproducing enhanced mediacontent; wherein the one or more initial preselected parameters formodifying the audio data are received without previously transmitting auser's parameters for modifying the audio data from the device; and aprocessor component configure to access the stored media contentincluding the audio data from the memory component and the informationabout the device for reproducing enhanced media content, access thefirst sound profile received by the receiver component, play the audiodata in the media content modified according to the first sound profile,and control the haptic device movement while playing the audio databased on the first sound profile.
 13. The device of claim 12, whereinthe haptic device comprises an attachment component to attach the hapticdevice to a contact point on a human body, a mechanical movementcomponent to generate a mechanical movement on the contact point. 14.The device of claim 12, wherein the first sound profile comprises datarelated to haptic movement of the audio data which is specific to a leftear or a right ear, demographic information, ethnicity information, ageinformation, location information, social media information, intensityscore of the audio data, previous usage information, and deviceinformation.
 15. The device of claim 12, wherein the processor componentis further configured to: determine a first inaudible frequency rangethat is inaudible for a user; compensate the first inaudible frequencyrange; update the first sound profile with the first inaudible frequencyrange for the audio data to become a second sound profile; and modifythe audio data in the media content according to the second soundprofile.
 16. The device of claim 12, wherein the compensating the firstinaudible frequency range for the audio data is performed by generatinghaptic movement corresponding to the first inaudible frequency range.17. The device of claim 12, wherein the processor component is furtherconfigured to: divide an audio frequency region into a number offrequency ranges; select a frequency range that is audible; decrease orincrease an audio volume at the selected frequency range until itbecomes a lowest audible volume in the selected frequency range; recordthe lowest audible volume for the selected frequency range; and updatethe first sound profile with the lowest audible volume for the selectedfrequency range.
 18. A device for reproducing enhanced media content,the media content including audio data, comprising: a haptic device witha sensor capable of being placed on a contact point on a human body; amemory component capable of storing media content, wherein the mediacontent includes audio data and audio metadata related to the audio datain the media content; a network component capable of transmittinginformation about the device for reproducing enhanced media content,audio metadata related to the audio data in the media content, and asound profile over a network, and capable of receiving a first soundprofile over the network, wherein the first sound profile is capable ofcontaining initial preselected parameters for modifying the reproductionof the audio data; and a processor component configured to: access thestored media content including the audio data from the memory component;access the first sound profile received by the network component; selecta frequency range; determine a lowest haptic intensity for hapticmovement that can be detected at the contact point for the frequencyrange; update the first sound profile with the lowest haptic intensityfor the frequency range to become a second sound profile; and modify thereproduction of the audio data in the media content according to thesecond sound profile.
 19. The device of claim 18, wherein the soundprofile comprises information identifying the location of the hapticdevice on a user's body.
 20. The device of claim 18, wherein theprocessor component is further configured to: determine a firstinaudible frequency range that is inaudible for a user; compensate thefirst inaudible frequency range under a first compensation parameter;update the second sound profile with the first compensation parameterfor the first inaudible frequency range to create a third sound profile;and modify the audio data in the media content according to the thirdsound profile.